Power Tools 1993 November - Disc 1/Power Tools Plus (Disc 1 of 2)(November 1993)(HP).iso/wkst/clrcswpp/clrcswpp.txt

Text File | 1992-06-04 | 27KB | 445 lines

HP/ClearCase
White Paper

This document presents an overview of HP/ClearCase. It presents the
central concepts in the product and an "executive summary" of the
product's features.

Introduction to HP/ClearCase
HP/ClearCase(TM) is a UNIX software engineering toolset for version
control, configuration management, and process control.
   HP/ClearCase is designed for large-scale and long-lived software
development projects. The architecture of HP/ClearCase is aimed at
providing these capabilities without changing the way a current
engineering environment works. It is targeted at teams of developers
working in a UNIX local area network. Versions of HP/ClearCase
running on different hardware platforms are fully compatible.
   HP/ClearCase manages multiple versions of evolving software,
tracks which versions were used in software builds, performs builds
and re-builds of individual programs or entire releases according to
user-defined version specifications, and enforces project-defined
development policies.
   These capabilities allow HP/ClearCase to address critical
requirements of organizations that produce and release software:
o Effective Development. HP/ClearCase enables developers to work
  ficiently as a team, allowing them to fine-tune the balance between
  sharing each others' work and isolating themselves from destabilizing
  changes. HP/ClearCase automatically manages the sharing of both
  source files and "derived" files (object modules, executables).
o Effective Management. HP/ClearCase provides complete tracking of
  the software build process, so that managers can determine what was
  built, and how it was built. Further, HP/ClearCase can instantly
  recreate the source base from which a software system was built,
  allowing it to be rebuilt, debugged, and updated -- all without
  interfering with other programming work.
o Enforcement of Development Policies. HP/ClearCase enables project
  administrators to define development policies and procedures, and
  automates their enforcement.

HP/ClearCase Feature Summary
o Enhanced Version Control. HP/ClearCase includes a version-control
  system that supports versioning of all file system objects: files,
  directories, and links. Any type of file can be versioned, including
  executables, bitmaps, and other non-ASCII files. Versions of
  directories record the evolution of the organization of the source
  base: renaming of source files, creation of new source files, and so
  on
o Transparent Access to Versions. Each versioned object appears to be
  an ordinary file or directory. No special command, such as SCCS's
  get, is required to access individual versions.
o Configuration Management: Complete Build Auditing. HP/ClearCase
  produces build audits called configuration records, which provide
  complete "bill-of-materials" documentation of software builds. Each
  configuration record includes a listing of all source file versions
  that were used and all build options that were specified. Special
  commands compare configuration records, showing the differences
  between two builds of the same program.
o Configuration Management: Rule-Based, Dynamic Version Selection.
  Users can create and use any number of views, each of which selects a
  particular configuration of source versions. Views are defined by
  configuration specs consisting of a few powerful, general rules --
  there is no need to specify hundreds or thousands of source versions
  individually. Views are dynamic -- HP/ClearCase continually updates a
  source configuration by reevaluating the rules that define it. Newly-
  created versions can thus be incorporated into a view automatically
  and instantly.
o Make-Compatible Software Building. Software build procedures are
  captured in standard UNIX makefiles. Developers can use any build
  tools, including make, to build software from HP/ClearCase-maintained
  sources.
o Build Tool Enhancements. HP/ClearCase's clearmake utility provides
  several optimizations, including the build auditing capability
  described above. Whenever possible, clearmake causes derived objects
  produced by builds to be shared among views. This eliminates the cost
  -- both in disk storage and in processor usage -- of creating
  additional, redundant instances of the objects. clearmake can execute
  several build scripts in parallel, on different hosts in the local
  area network.
o Guaranteed Rebuildability. A view's configuration spec can be
  defined in terms of the configuration records produced by previous
  builds. The exact source base for an individual program or an entire
  release can be recreated instantly in a new view, thus guaranteeing
  rebuildability.
o Additional Development Tools. HP/ClearCase supplements UNIX's rich
  development toolset with a multi-file text-comparison utility and a
  multi-file text-merge utility. These tools greatly improve efficiency
  in environments where several variants of a program are being
  developed concurrently.
o Conservation, Distribution, and Scaling of Resources. HP/ClearCase
  minimizes data duplication, both for source files (elements) and for
  build targets (derived objects). A source file version is copied only
  when a developer wishes to edit it. If a build script would create a
  redundant copy of an object module or executable, HP/ClearCase
  instead creates a link to an existing instance.

Architecture
Versions of text files are stored efficiently as "deltas", much like
SCCS or RCS versions. Versions of non-text files can also be stored
efficiently, using data compression.
   Source data is stored in mountable filesystems, called versioned
object bases (VOBs). VOBs can be physically located throughout the
local area network, and mounted on some or all workstations. Further,
the data storage for an individual VOB can be distributed across the
network, even to hosts that are not running HP/ClearCase software.
   HP/ClearCase's client-server architecture optimizes usage of
network resources and provides scalability. Typically, VOBs
containing shared data (and associated server processes) are located
on high-speed network server hosts. Developers and their views are
hosted on individual workstations. As additional developers (and
their workstations) are added to a project, more compute and storage
resources become available for HP/ClearCase usage.

Version Control
Information in the software development environment is constantly
being changed. The changes are motivated by shifting requirements and
the general need for improvement and software maintenance. Older
versions of the software development elements are needed to recreate
complete versions of working software previously used. Newer versions
of these elements are necessary to both back-out of mistakenly
followed development paths, and to allow members of a development
team to avoid constantly shifting sand by select an individual
version of a software component to use even while additional new
versions are being created.
   Source control systems, such as SCCS and RCS, capture the changes
made to text files by providing a facility to efficiently store new
versions and to maintain the evolutionary relationships among them.
These, and similar systems also provide version annotation facilities
that support a change audit trail (i.e. who did what when and why)
and allow the attachment of easily understood labels (e.g RLS_2.1) to
mark individual versions for later retrieval.
   Version control systems use explicit commands to create new file
version trees, to add branches to these trees, and to add versions to
the branches. Such commands (e.g. check-out, check-in) regulate the
growth of the version trees, arbitrate conflicting parallel
developments, and help collect the information included in the change
history.
  Even in modest size projects, the version tree of a source code
file can readily grow to include many versions on several separate
development branches.
   HP/ClearCase, like SCCS and RCS, can maintain many versions of
each source file. Unlike other systems, HP/ClearCase can also
maintain versions of directories. All version-controlled data is
stored in mountable filesystems, called versioned object bases
(VOBs). A VOB is a network-wide resource -- hosts can mount any
number of VOBs with the standard UNIX mount command (Figure 0-1).
   Each version-controlled file in a VOB is called a file element;
its versions are organized into a version tree structure, with
branches and sub-branches (Figure 0-2). HP/ClearCase can version-
control any type of file, not just ASCII text files.
   Version trees store "old" versions of source files, enabling the
rebuilding and maintenance of "old" releases. Each (sub-)branch can
grow independently, so that multiple variants of a source file can be
developed concurrently.
   Each directory in a VOB is a versioned object called a directory
element. A version of a directory element is essentially similar to a
standard UNIX directory -- it lists file elements, (sub)directory
elements, and links. Successive versions of a directory element
record changes to the organization of the source code base, rather
than changes to the contents of individual files. This includes
creation of new files, renaming of files, even wholesale source tree
reorganizations.
   Although a VOB contains many versions of each file and directory,
most software "sees" the VOB as a standard UNIX directory tree. This
important feature is called transparency -- the version-control
system becomes invisible, or transparent. In each HP/ClearCase view,
a file element appears to be a ordinary file, and a directory element
appears to be an ordinary directory. This is accomplished by the
automatic selection of one version of each element. (A per-view
configuration file determines which version the view selects, as
described in the next section.)
   Transparency is implemented at the UNIX fie system level by a
HP/ClearCase virtual file system extension. Standard I/O calls (for
example, read(2) and open(2)) are intercepted in the UNIX kernel;
HP/ClearCase's version-selection mechanism reroutes the destination
of the call from an element to a selected version of that element.
   Transparency a key feature, because it enables HP/ClearCase to
work smoothly with UNIX system software, commercial applications, and
a user's in-house software tools. Users do not have to discard their
accustomed ways of working, or their existing tools.
   Users can override transparency, using version-extended pathnames
to access any version of an element explicitly (Figure 0-3).
   HP/ClearCase includes a set of commands for evolving elements. The
basic checkout-edit-checkin paradigm is similar to that used by
"traditional" UNIX version-control facilities, such as SCCS and RCS.
In addition to checkout and checkin, HP/ClearCase command set
includes delete version, create/delete branch, list version history
(chronological), list version history (structural), compare with
predecessor version, merge parallel versions, and many more.

Configuration Management -- Views
Version control facilities manage many versions of each file system
element. In HP/ClearCase, this management extends beyond just source
code to every file system object type and the even the file system
directories themselves. However, software development tools, and the
people who use them, work on individual files - not on collections of
versions of files. At any one time, for any one project, each
developer decides which versions form the appropriate development
starting point, and works just with those.
   The primary goal of a software configuration management system is
to locate the desired information and present it in a usable form. In
conjunction with a version control system, this requires powerful,
flexible and intuitive version selection to help developers identify
the specific versions with which they need to work. Once the proper
versions are selected, they must be presented to the user in a
familiar way, and in a form readily usable by standard software
development tools.
   Each developer requires a work environment that provides access to
the appropriate versions of source files, and which allows the
developer to work in a way that does interfere with (or suffer
interference from) other developers. The HP/ClearCase configuration
management mechanism, the view, provides both these services.
   A view is a working context for an individual developer or a
closely-coordinated group. Views implement HP/ClearCase's transparent
access feature, described in the preceding section. A view makes a
VOB appear to be a standard directory tree by selecting (at most) one
version of each element (Figure 0-4). One view might select the most
recent version of every element; another view might select the
versions used to build Release 2.4.3; still another view might select
the versions being used to fix a bug in Release 3.0.1.
   Each user can have many views, and can attach any window (that is,
any process) to any view.
   The set of versions selected by a view is called its
configuration. View configurations are dynamic, and can be modified
by the developer at any time. A view's configuration is defined by a
short, powerful set of rules, called a configuration specification
(config spec). Each time a program performs an open(2) call on an
element, the UNIX kernel uses the view to which the current process
is attached. The view_server process for that view consults the rules
in its config spec, decides which version of the element to select,
and returns this information to the kernel.
   Thus, a view is unlike a traditional "sandbox", which configures a
source tree by copying particular versions from a repository. A view
is not a set of files; rather, it is a way of "seeing" shared
elements. Each view can access any and all VOBs mounted on the
developer's host. When new versions or elements are added to a VOB,
or new VOBs are mounted, that data is immediately accessible to all
views. Developers can change their config spec rules interactively,
to balance the need for isolation from other views' source code
changes with the need to share such changes.
   Typically, a view's configuration is defined in terms of wildcards
(filename patterns) and mnemonic names, not by naming specific
versions. For example, the user-defined version label "REL2.4.3" can
be attached to versions of many different source files; and a branch
with the user-defined branch name "fix2.4.3" can be created in some
or all those file elements. A config spec with just two rules creates
a view with this configuration:"for all source files, use the most
recent version on branch fix2.4.3, or else use the version labeled
REL2.4.3".
   To facilitate rebuildability, a derived object can be used to
define a view's configuration directly, for example, "select the same
source versions that I used to build this executable last week".
   Each developer can have several views, used for different tasks:
new development, fixing a bug in a recent release, preparing a
special variant for a valued customer, porting to a new window
system, and so on. In each such view, the developer must be able to
edit source files, compile the software, and test it. To enable such
work to proceed without disturbing the work going on in other views,
each view has its own storage area. This view-private storage is used
for working versions of source file elements, text-editor backup
files, cut-and-paste temporary files, and so on. A developer's builds
are also placed in view storage, so that they do not interfere with
others' builds.
   For example, two developers in different views might edit the same
source file and rebuild the same executable, without getting in each
others' way. (In fact, they might help each other by being in the
same place at the same time -- see the next section.)
   HP/ClearCase views are designed to be "cheap" -- they can be
created quickly and easily, are easy to modify, and require minimal
system resources. For example, a minor variant of a 5000-file source
configuration might be required to create a customer-specific version
of a software system. The view in which this task will be performed
can be created in ten seconds, by editing a few version-selection
rules. The disk requirements for this view are small -- enough space
to store modified source files, corresponding object modules, and the
executable(s) to be built. Most sources and object modules need not
be copied or rebuilt. Unmodified sources are accessed from VOBs;
object modules corresponding to unmodified sources are reused from
other views.
   Both views and VOBs are network-wide resources, accessible through
NFS services. This facilitates team-oriented development, and enables
data managed by HP/ClearCase to be used on hosts where HP/ClearCase
software has not been installed.

Edit/Compile/Debug Cycle
HP/ClearCase is compatible with standard UNIX development tools and
practices. It is easy for organizations to make the transition to
HP/ClearCase usage; in particular, the transition can be incremental,
some groups making the switch before others. HP/ClearCase includes
tools for importing version-controlled files from other systems, such
as SCCS and RCS.
   The following outline of the typical development cycle
demonstrates HP/ClearCase's compatibility, while highlighting its
"value-added" features.
o To start working, a developer creates a shell process that is
  "attached" to a view, then uses the standard cd command to visit any
  source directory, in any VOB. There is no need to copy data to a
  "parallel source tree".
o All of the developer's shell-oriented tools (aliases, filename
  completion, command-output substitution, scripts, and so on) remain
  available as the developer works on HP/ClearCase data.
o The developer cannot modify a source file element until it is
  "checked out". An element appears as a read-only file until the
  developer issues a checkout command to make it writable. Only the
  developer (and other users using the same view) can access the
  writable, checked-out version.
o The developer can build the software system with the standard
  make(1) utility, with HP/ClearCase's enhanced clearmake build
  utility, or with any other UNIX facilities (for example, shell
  scripts).
o clearmake is compatible with standard make, but offers a powerful
  set of additional features:
  - Build Auditing. For each build script it executes, clearmake
    stores a configuration record (config rec) that details the build
    process. The config rec includes "who, what, when, where"
    information, along with a list of exact source file versions used,
    a copy of the build script used, and any build options specified on
    the command line or in the environment.
       HP/ClearCase automatically associates the config rec with each
    object produced by the build script (for example, an object module
    and a compiler listing file). These derived objects are
    automatically  tracked by HP/ClearCase.
  - Derived Object Sharing. Instead of invoking a build script to
    create a target, clearmake can instead find an appropriate derived
    object that was built in another view. It creates (in essence) an
    additional hard link to the derived object, causing it to be shared
    by the views. This capability saves both disk storage and build
    time.
  - Parallel, Distributed Builds. When build dependencies allow,
    clearmake executes multiple build scripts in parallel. The parallel
    executions can take place on a single multi-processor host and/or on
    several hosts throughout the local network.
o The developer can use any debugger (for example, dbx(1) or a
  vendor-supplied tool) to analyze and fix bugs in built software. The
  correct source file versions are all accessible through the view.
o The checkin command creates a new version of an element. The new
  version instantly becomes available to all views, and visible in
  views that have been configured to "see" the latest version of that
  element.
o Developers can monitor and compare each others' edits and builds
  with special HP/ClearCase commands: list checkouts, list derived
  objects, list/compare config recs, compare source versions, and so
  on.

Process Management / Policy Enforcement
In addition to providing tools that control sources and derived
objects, HP/ClearCase provides tools for monitoring and controlling
the development process itself. These tools rely on HP/ClearCase
meta-data, information that pertains to, but is separate from, the
actual contents of files and directories. The preceding section
introduced two kinds of meta-data: version labels and branch names.
Additional kinds support the definition and enforcement of process
management policies that go beyond standard UNIX access controls.
o Attributes. An attribute is a name=value pair, similar in form to
  an environment variable. A generalization of version labels,
  attributes can be assigned to a wide variety of objects (for example,
  derived objects), not just versions of elements; and they can have
  strongly-typed values, such as integers in specified ranges.
o Hyperlinks. A hyperlink is a logical "arrow" (one-headed or two-
  headed) that connects two VOB objects. Either end of the "arrow" can
  be annotated with a text string.
o Triggers. A trigger is a "flag" that monitors the development
  process, and automatically responds to various commands executed by
  developers. A trigger can disallow certain commands from being
  executed.
   HP/ClearCase does not attempt to dictate any particular process
management policies or procedures. Rather, it supplies these meta-
data facilities as a flexible, powerful "toolset". Administrators can
use these tools to automate the enforcement the organization's
existing policies.
   At the simplest level, triggers enable automation of vital
communications links among members of the development team (Figure 0-
5).
   At a more advanced level, triggers are the enforcement mechanism
required for a comprehensive and robust process management system. A
trigger can interrupt the action that triggered it (for example, a
checkin command), to determine whether the action conforms to the
organization's development policies. If the action is not valid, the
trigger might cancel the action; the response to a valid action might
be to attach a version label or attribute to the relevant files.
   For example, the QA Department might be mandated to analyze the
degree to which each source module complies with company coding
standards. In a partially automated system, every checkin of a source
file might trigger sending of mail to a QA engineer. The QA engineer
might then determine the compliance level and assign attribute
CodeLevel to that source file version, with the value "not
compliant", "minimally compliant", or "optimally compliant".
   In a more fully automated system, a checkin command might trigger
the automatic running of a compliance script. Such a trigger could
attach the CodeLevel attribute to the checked-in version, or cancel
the checkin command if the code is not compliant.
   Hyperlinks allow interrelationships among different aspects of a
project to be defined, tracked, and controlled. For example,
"requirements tracing" for a particular proposal might be implemented
with a set of hyperlinks, all of which connect the proposal document
to associated design notes and source files. Each hyperlink from the
proposal document might be annotated with a reference to the specific
section to which the design note and source file corresponds (Figure
0-6).

Release Management
Source control relates to the files that go into software builds;
release management relates to the files that are produced by builds,
and are subsequently placed on release tapes (or other release
media).
   During development, many versions of derived objects are produced.
Most of these have a short lifetime -- developers continually revise
sources and rebuild. Eventually, a "good" build is deemed ready for
release (by the developer, a manager, the QA Department, and so on).
At this point, a derived object can be "released" to a more permanent
status by checking it in as a version of an element. It can then be
treated like a source file.
   As with process management, HP/ClearCase does not dictate a
particular mechanism for defining and producing product releases. But
HP/ClearCase's ability to version-control any kind of file makes it
easy to apply source control techniques to the challenge of release
management.
   The Release Engineering group might maintain one or more "release
tree" VOBs. The directory structure of the tree(s) mirrors the
hierarchy of files to be created on the release medium. (Since a
release tree involves directory elements, it is easy to change its
structure from release to release.)
   A release tree might be used to organize Release 2.4.3 as follows:
o When an executable or other file is ready to be released, a release
  engineer checks it in as a version of an element in the release tree.
o An appropriate version label (for example, REL2.4.3) is attached to
  that version, either manually by the engineer or automatically with a
  trigger.
o When all files to shipped have been released in this way, a release
  engineer configures a view to select only versions with that label.
  As seen through this view, the release tree contains exactly the set
of files to be released.
o To cut a release tape, the engineer issues a simple tar(1) or
  cpio(1) command to copy the appropriately-configured release
  tree.fooc.@@/main/bugs/bug404/

UNIX is a registered trademark of UNIX System Laboratories, Inc. in
the USA and other countries.OSF/Motif is a trademark of the Open
Software FoundationClearCase is a trademark of Atria Software, Inc.

Copyright (C) Hewlett-Packard Company, 1992
Printed in USA 920601

HP/ClearCase White Paper

Associated Files: none